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Abstract. We study mixing properties of commutative groups of automorphisms acting 
on compact nilmanifolds. Assuming that every nontrivial element acts ergodically, we 
prove that such actions are mixing of all orders. We further show exponential 2-mixing 
and 3-mixing. As an application we prove smooth cocycle rigidity for higher-rank abelian 
groups of nilmanifold automorphisms. 



Given a measure-preserving action of a (discrete) group T on a probability space (X, /x) , 
we say that this action is (s + l)-mixing if for every fo,...,f a £ L°°(X) and 70, ■ ■ ■ ,J S £T, 



as 7i!7j~ — > 00 for all i\ 7^2- In particular, 2-mixing corresponds to the usual notion of 
mixing. It was discovered by F. Ledrappier |12] that 2-mixing does not imply 3-mixing in 
general. In this paper we will be interested in mixing of higher order. This property is 
a very widespread phenomenon for one-parameter actions. It is known to hold for many 
transformations satisfying some hyperbolicity assumptions. On the other hand, very little 
is known about higher order mixing for actions of large groups. We are only aware of 
two general families of actions of large groups on manifolds where the multiple mixing 
has been established — Z'-actions by automorphisms on compact abelian groups and 
actions of simple Lie groups. K. Schmidt and T. Ward [18] proved that 2-mixing Z^-action 
by automorphisms on compact connected abelian groups are mixing of all orders, and 
S. Mozes [13] established mixing of all orders for ergodic actions of simple Lie groups. 

In this paper we investigate mixing properties of Z'-actions by automorphisms on com- 
pact nilmanifolds. We prove that for such actions, 2-mixing implies mixing of all orders 
and establish quantitative estimates for 2-mixing and 3-mixing. 

1.1. Main results. Let G be a simply connected nilpotent group and A a discrete co- 
compact subgroup. We call the space X = G/A a compact nilmanifold. We denote by 
Aut(X) the group of continuous automorphisms a of G such that a (A) = A. Then Aut(X) 
naturally acts on X and preserves the Haar probability measure fi on X. 

Our first main result concerns exponential 3-mixing. In order to obtain any quantitative 
estimate in (jl.ip . it is necessary to work in a class of sufficiently regular functions. We 
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A 



denote by C e (X) the space of Holder functions with exponent 6, defined with respect to 
a Riemannian metric on X. 

Theorem 1.1. Let a : Z l -> Aut(X) be an action on a compact nilmanifold X such 
that every a(z), z ^ 0, is ergodic. Then there exists r\ = n{6) > such that for every 
fo, fi, h G C (X) and z , Zi,z 2 G Z l , 

fo(a(zo)x)fi(a(z 1 )x)f 2 (a(z 2 )x) dfx{x) 

J Q d^j (7 (f f* d ») + 0{N{z 0> Z 1 ,Z 2 )-^fo\\c»\\fl\\c'\\f2\\c')> 

where N(zq, zi, z 2 ) = exp(minj^j \\zi — Zj\\). 

We note that this result is new even for the case of toral automorphisms. Mixing of all 
orders for ergodic commuting toral automorphisms was established in |18] . The argument 
in |18j relies on finiteness of the number of nondegenerate solutions of S"-unit equations 
established in |15j . Although there are explicit estimates on the number of such solutions, 
these estimates are not sufficient to derive any quantitative estimate for 3-mixing because 
it is also essential to know how the sets of solutions depend on the coefficients. In order 
to prove Theorem ll.l[ we use more delicate Diophantine estimates for linear forms in 
logarithms of algebraic numbers established in [20] (cf. Proposition 12.21 below). 

We also prove mixing of all orders: 

Theorem 1.2. Let a : Z l Aut(X) be an action on a compact nilmanifold X such that 
every a(z), z 0, is ergodic. Then for every fo,...,f 3 € L°°(X) and zq, ■ ■ ■ , z s € l} , 




f[fMzi)x)) dfi(x) = n (7 fidA +o(l) 



=0 / i=0 

as minj^j \\zi — Zj\\ —¥ oo. Moreover, the convergence is uniform over families of Holder 
functions f , ■ ■ ■ , f s such that ||/o||c e ) ■ ■ ■ > \\fs\\c e < 1 - 



This theorem extends the main result of [18] to general nilmanifolds. The proof in 
|18j utilises abelian Fourier analysis and properties of solutions of 5-unit equations. Our 
approach is based on the study of distribution of images of polynomial maps in X. This re- 
duces the proof to the investigation of certain Diophantine inequalities which are analysed 
using W. Schmidt's Subspace Theorem. In order to prove an effective version of Theorem 
11.21 one would need to estimate the size of nondegenerate solutions of these Diophantine 
inequalities in terms of complexities of coefficients (cf. Proposition 13.11 below). However, 
this seems to be far out of reach of available techniques when s > 2. 

Finally, we discuss the problem of exponential mixing for shapes in Aut(X). This notion 
was introduced by K. Schmidt in [16] in order to better understand Ledrappier's examples 
|12| which are not mixing of higher order. A shape in Aut(A) is a collection of elements 
cto, • • • , a s G Aut(A). We says that the shape is mixing if for every /o,...,/ 4 G L°°(X), 




OL; x 



dfi(x) — > f[ fi di^j 
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as n — > oo. This property has been extensively studied in the context of commuting 
automorphisms of compact abelian groups (see, for instance, [6], [17^ Ch. VIII], [22], [23]). 

We establish quantitative mixing for commuting Anosov shapes. We say that the shape 
ceo, •••,is is Anosov if a^a" 1 is an Anosov map for all i\ ^ i^. 

Theorem 1.3. Let X be a compact nilmanifold and ao,...,a s € Aut(X) a commuting 
Anosov shape. Then there exists p = p{9) € (0, 1) such that for every /o, . . . , f s € C S (X) 
and n G N, 

(i-2) J x (ufi(< x yj w=n^/ i d / i)+o^nii/ i ib^. 



1.2. Applications to rigidity. The exponential mixing property played an important 
role in the program of classification of smooth Anosov higher-rank Z'-actions on compact 
manifolds. It is expected that all such actions can be built from actions by automorphisms 
on nilmanifolds. D. Fisher, B. Kalinin and R. Spatzier in [7] applied the exponential 
2-mixing property to extend their results for Anosov Z'-actions on tori to actions on 
nilmanifolds: 

Theorem 1.4 (D. Fisher, B. Kalinin, R. Spatzier). Let a be a C 00 -action of ' l) , I > 2, 
on a compact nilmanifold X and let p : l) — > Aut{X) be the map induced by the action of 
a(Z ) on the fundamental group of X. Assume that there is a Z 2 subgroup ofl) such that 
p(z) is ergodic for every nonzero z G Z 2 , and there is an Anosov element for a in each 
Weyl chamber of p. Then a is C°° -isomorphic to p. 

In fact, this application to global rigidity was our original motivation to establish the 
exponential mixing property for nilmanifold automorphisms. 

We also use the exponential mixing property to establish cocycle rigidity for higher- 
rank Z'-actions by automorphisms of nilmanifolds, extending the results of A. Katok and 
R. Spatzier |1CH 111] , A C°° -cocycle is a C°°-map c : H) x X \— > R satisfying the identity 

c{z\ + Z2, x) = c(zi, Z2 ■ x) + c(z2,x) for z\,Z2 G Z* and 

Two cocycles c\ and C2 are called smoothly cohomologous if there exists b £ C°°(X) such 
that 

c\(z, x) = C2(z, x) + b{z • x) — b(x) for z € Z' and 

We call a cocycle constant if it does not depend on x € X. We prove that cocycles over 
genuine higher-rank actions by automorphisms on nilmanifolds are smoothly cohomologous 
to constant cocycles. This phenomenon was first discovered by A. Katok and R. Spatzier in 
[10] for certain higher-rank Anosov actions. Using our methods, we generalise this cocycle 
rigidity theorem to actions by automorphisms on nilmanifolds. We emphasise that the 
action in the following theorem need not be Anosov. 

Theorem 1.5. Let a : Z l -> Aut(X) be an action on a compact nilmanifold X . Assume 
that there is a Z 2 subgroup ofll such that a(z) is ergodic for every nonzero z G Z 2 . Then 
every smooth It-valued cocycle is smoothly cohomologous to a constant cocyle. 
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For certain actions by partially hyperbolic left translations on homogeneous spaces 
G/T, where G is a semisimple Lie groups and T is a lattice in G, a similar theorem was 
proved by D. Damjanovic and A. Katok [H[5l[3] and Z. J. Wang [21]. We note that these 
authors also prove Holder versions of this result which are not amenable to our techniques. 
Furthermore, cocycle rigidity results are proven for small perturbations of these actions 
on G/T in [5j[2T]. Again we cannot obtain these results by our methods. 

Acknowledgements. A.G. would like to thank the University of Michigan for hospitality 
during his visit when the work on this project had started. R.S. thanks the University of 
Bristol for hospitality and support during this work. 

2. Exponential 3-mixing 

In this section we prove Theorem ll.il We start by setting up basic notation, which will 
be also used in subsequent sections. Then in Sections I2.2H2.41 we collect some auxiliary 
estimates. The proof of Theorem II .11 is divided into two parts. We first give a proof under 
an irreducibility condition in Section 12. 5\ and then in Section 12.61 prove Theorem 11.11 in 
general using an inductive argument. 

We note that if the reader is only interested in exponential 2-mixing, then the results 
of Section [2.31 are not needed, and in Section [2.51 one only needs to consider Case 1. This 
makes the proof much simpler. 

2.1. Notation. Let G be a connected simply connected nilpotent Lie group, A a discrete 
cocompact subgroup, and X = G/A the corresponding nilmanifold equipped with the 
invariant probability measure (M. We fix a a right invariant Riemannian metric d on G 
which also defines a Riemannian metric on X. Let C(G) be the Lie algebra of G and 
exp : C{G) —¥ G the exponential map. The lattice subgroup A defines a rational structure 
on C{G). For a field K D Q, we denote by C(G)k the corresponding Lie algebra over 
K. Let 7r : G —¥ G/G' denote the factor map, where G' is the commutator subgroup. 
We also have the corresponding map Dir : C(G) — > C{G/G'). We fix an identification 
G/G' ~ C(G/G') ~ R d that respects the rational structures. 

Every automorphism (3 of G defines a Lie-algebra automorphism D(3 : C{G) — > C(G) 
such that f3 o exp = exp oDfi. If /3(A) = A, then D/3 preserves the rational structure of 
C(G) defined by A. In particular, given an action a : 7l — > Aut(A) on the nilmanifold 
X = G/A, we obtain a homomorphism Da : l) — > GL(£(G)q). 

For a multiplicative character x : l) — >■ C x , we set 

C x := {u G C(G) ® C : Da(z) u = x(z)u for z G Z 1 }. 

Let X{a) denote the set of characters x appearing in the action Da on C(G), and X'(a) C 
X{a) be the set of characters appearing in the action on C(G)/C{G)' . 

2.2. Estimates on Lyapunov exponents. Since a(Z') preserves the rational structure 
on C(G) defined by the lattice A, it follows that each character x m X{a) is of the form 
x(z) = u^ 1 ■ ■ ■ Ui 1 where u^s are algebraic numbers. The Galois group Gal(Q/Q) naturally 
acts on X (a) and X'(a). Let Xq C X'(a) be one of the Galois orbits. 
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Lemma 2.1. Suppose that every a(z), z ^ 0, acts ergodically on X. Then there exists 
c > such that 

max |x( z )| > ex P( c ll- 2; ll) f or a M z ^l) . 
xe<%b 

Proof: By [21 5.4.13], AG' /G 1 is a lattice in G/G' , and the action a defines the action on 
the torus T := G/(AG') by linear automorphisms. Let V be the subspace of C(G)/C(G)' 
spanned by the x- e ig ens P aces i n with x G %o- Clearly, this subspace is invariant under 
a(Z') and is defined over Q. Hence, it defines an a-invariant subtorus T% of T. Since 
cx{z)\t is ergodic when z ^ 0, it follows that the corresponding linear map has no roots of 
unity as eigenvalues. This implies that o.(z)\t Xq is also ergodic. 

Consider a linear map £ : M l — > W x °\ which is defined for z € Z' by 

£(z) := (log : xG^o) 

and extended to R by linearity. Since for every z € Z'\{0}, the automorphism a(z) acting 
on Tx is ergodic, we have £(z) ^ by [£l Lemma 3.2]. Hence, £\ z i is injective. 

We also claim that 1(7}) is discrete. We consider the embedding l) — > GL(V) defined 
by a. Since a(Z') preserves the integral lattice in V corresponding to the torus T^ , it 
follows that the image of this embedding is discrete. In other words, the subset 

of (C x )\ x °\ is discrete. Since the kernel of the natural homomorphism (C x )' —¥ W 
defined by s i— ?■ log |s| is compact, this implies that £(Z') is discrete, as claimed. Since 
^(Z') is discrete and has rank I, it follows that the space £(M. 1 ) has dimension I, and, in 
particular, the map £ is injective. Therefore, by compactness, there exists c > such that 
for every z € Mf, we have 

max{log |x0)| : X G X } > c\\z\\. 

This implies the lemma, o 



Lemma 12. II shows that in Theorem II .11 we may replace N(zq, z\, z-i) by 

exp I min max \x( z i — Zj)\ ) ■ 

2.3. Diophantine estimates. Recall that the (absolute) height of an algebraic number 
u is defined by 

/ \ l/[Q(u):Q] 

H(u) = I JJmax(l, \u\ v ) J , 

where | • \ v denote suitably normalised absolute values of the field Q(u). When u is an 
algebraic integer, the height can be computed as 

l/[Q(w):Q] 



H(«) = ^max(l,|«i|)J 



where Uj denote all the Galois conjugates of u. 

The following result is deduced from the work of M. Waldschmidt \20\ Cor. 10.1]. 
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Proposition 2.2. Let u\, . . .ui,u £ C be algebraic numbers and z = (z\, . . . , z{) € ll . 

Then there exists 01,02,03 > 1, depending onu\, . . . ,u\ and [Q(u) : Q], such that assuming 
that 

(2.1) ||z|| > log(c 2 tf(u)), 

and 

■ --ufu ^ 1, 

we have the estimate 

c 3\\ z \\ 



(2.2) {ul 1 ■■■u l l u - 1| > exp -cilog(c 2 #(u))log . 

V Vog{c 2 H(u)) 

Surprisingly, it turns out that the term log(c2H(u)) in the denominator is essential to 
establish exponential 3-mixing (cf. (I2.28j) - (|2.30|) below). 

Proof: We note that since H(u) > 1 and (|2.ip holds, the right hand side of (|2.2p is 
bounded from above by 

exp(-ci log(c 2 )log(c 3 )). 
Taking the constants sufficiently large, we may arrange that this quantity is bounded by 
1/2. Then (|2.2p trivially holds when ■ ■ -uf l u — 1| > 1/2, and without loss of generality 
we assume that \u z ^ ■ ■ ■ u Zl u — 1| < 1/2. 

Let log denote the principle value of the (complex) logarithm. There exists zq € Z such 
that \zq\ <C ll^ll and 

T := log(uf ■■■u z £u) = Trizo + z\ log u\ -\ V z\ log u\ + log u. 

It is convenient to set u$ = —1, so that log no = vri. Let S := u* 1 ■ ■ ■ u Zl u. Since \S — 1| < 
1/2, 

\T\ = I log ^1 < 2|5-1|. 
Hence, it sufficient to establish a lower bound for \T\. Note that since S 7^ 1, we have 
T / 0. For this purpose we use |20| Cor. 10.1], which we now recall. We note that the 
result in [20] is stated using the logarithmic height while here we use the exponential 
height. For simplicity, we take E = e and / = 1. 

Let D = [Q(tio, ■ ■ ■ ,ui,u) : Q], Aq, . . . , A[, B be numbers, greater than e, such that 

H(ui)<Ai, i = 0,...,l, H(u)<B, 



i=0 



We set 



A = max{A , . . . ,Ai,B}, 

f 1 I* 
M = max <^ + 



i=o,...,i I log Ai log B 
Z = max{7 + 31og(Z + 2),logL>}, 
G = max{4(/ + 2)Zo,logM,logL>}, 

17 = max [d 2 log A, D i+4 G Z (log A ) ■ ■ • (log A,) (log B) } . 
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Then according to [201 Cor. 10.1], 

(2.4) |T| >exp(-c*7 ), 

where c is an explicit positive constant depending only on n. We set B := C2^{u) with 
C2 > 1. We note that 

I log u| 2 < vr 2 + log 2 \u\ < vr 2 + [Q(u) : Q} 2 log 2 H(u). 

Therefore, taking ylj sufficiently large, depending on Uj, and sufficiently large C2, we may 
arrange that f|2.3j) holds. If C2 is sufficiently large, A = B. Under the assumption (|2.ip . we 

have M < with sufficiently large C3 and also Go = logM < log ( j . Moreover, if 
C3 is sufficiently large, then Uq <C log j logi?. Therefore, estimate (|2.4p implies that 

|r|>ex P (- Cl log (gM) log B 
where c\ is an explicit positive constant. This completes the proof of the proposition, o 



2.4. Equidistribution of box maps. A box map is an affine map 

t : B := [0,Ti] x ••• x [0, T fc ] -> 

of the form 

(2.5) 1 : (ii,... ,t k ) t-+v + hwi H htfc^fc 

with u, toi, . . . , € C{G). We shall use the following result, which is a variation of our 
theorem Th. 2.1], that implies equidistribution of box maps under suitable Diophantine 
conditions. This result is based on the work of B. Green and T. Tao [8]. 
We denote by \B\ the /c-dimensional volume of the box B. 

Theorem 2.3 (Green-Tao dichotomy). Fix < 9 < 1. There exist L±,L2 > such that 
for every 6 € (0, So) and every box map 1 : B — s> C{G), one of the following holds: 
(i) For every 6 -Holder function f : X — > M, u £ £>(G), and g £ G, 



Pi X ^ exp ^ ex P( i ( t ))5 ,A ) dt- J fdfi 



<s\\f\\c- 



(ii) There exists z 6 Z d \{0} smc/i i/iai 



and 



1. 



,fc. 



{z,Dir(wi)) I < 5~ L2 /Ti for alii 

Proof: In the case of Lipschitz functions /, this is [£l Th. 2.1], and the analogous result 
for Holder functions can be deduced by a standard approximation argument. Indeed, 
suppose that for some / € C (X), u 6 C(G), and g £ G, 

W\ Ib J ^ 6XP ^ exp ^(*)^ A ) dt- J^fd/J, 
Then one can find a Lipschitz function / E such that 

||/ £ -/||c°<^ll/llc* and ||/ £ ||L ip «e- dim W- 1 l 



(2.6) 
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(see, for instance, [U Lem. 2.4]). Then taking e = (<5/3) 1 / e , we deduce from (j2.6[) that 



Now the theorem for Lipschitz functions implies that (ii) holds with some L\,L2 > 
depending on 8. o 



2.5. 3-mixing under irreducibility condition. The action of Da^L ) preserves the 
rational structure on C{G) defined by the lattice A. In particular, it follows that each 
character x m X(a) is of the form x( z ) = uZ \ ' ' ' u i l where UiS are algebraic numbers. 
The Galois group Gal(Q/Q) naturally acts on X(a) and on C(G)q. We fix an orbit 
Xq C X'{a) of the Galois group and for each x £ <^Cb we fix a vector w x € C x whose 
coordinates are algebraic integers, so that the vectors w x are also conjugate under the 
action of the Galois group. Let Wc be the Lie subalgebra of C(G) (g> C generated by the 
vectors w x , x £ %o-, and W = Wc H £(G). We also fix a basis {wj} of W. 

In this section, we prove Theorem 11.11 under the irreducibility assumption: Dtt(W) is 
not contained in a proper rational subspace. Let w x = Dtt(w x ), x £ ^b- We observe that 
under this assumption the coordinates of each the vectors w x are linearly independent 
over Q. Indeed, if we suppose that {a,w x ) = for some a € Q d \{0}, then applying the 
action of the Galois group, we deduce that (a,w x ) = for all % e ^o- Since Dir(W) is 
spanned over C by the vectors w x , x £ Xo, this would imply that Dir(W) is contained in 
a proper rational subspace, which contradicts our assumption. 



Without loss of generality, we may assume that zq = and iV = exp(||zi — Z2\\)- We set 
e = N~ K , where k > is a fixed parameter which is sufficiently small and will be specified 
later (see 422U>, flZZgj) , p^H]) . <|CT]> below). 

We fix a fundamental domain F C G for X = G/A and set £7 = exp _1 (i ? ). As in [9J 
Section 3], we may arrange that E is bounded and has piecewise smooth boundary. Since 
the Haar measure on G is the image under exp of a suitably normalised Lebesgue measure 
on C(G) IU 1.2.10], we obtain 




^e di ^ + \5-2e 9 )\\f £ \\ Lip 

>>5 (di m (A) + l) /e+ l|| /e ||^. 



Let 



(2.7) 




(2.8) 





E 



fo(exp(u)A) fi(exp(Da(zi)u)A) f2(exp(Da(z2)u)A) du. 
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We choose a basis of C(G) that contains the basis {wi} of W and tessellate C(G) by cubes 
C of size e with respect to this basis. Since E has piecewise smooth boundary, we obtain 



(2.9) 



and 



E-[JC 

CcE 



(2.10) / fo(exp( U )A)f 1 (exp(Da(z 1 )u)A)f 2 (exp(Da(z2)u)A)du 



E 



^2 / fo(exp(u)A)f 1 (exp(Da(zi)u)A)f 2 (exp(Da(z 2 )u)A)du 

CcE^ C 

+ O(e|l/o|lc £ 'll/illc e ll/2||c e )- 



For every cube C, we pick a point uq £ C. Then since /q is #-Hblder, 

(2.11) / Mexp(u)A)f 1 (exp(Da(zi)u)A)f 2 (exp(Da(z 2 )u)A)du 

Jc 



=/o(exp(uc)A) / fi(exp(Da(z 1 )u)A)f 2 (exp(Da(z 2 )u)A) du 
Jc 

+ O(e 6 '||/o|| c e||/i|| c .9 1|/2 He" )• 



We decompose each cube C as C = B' + B, where B is a cube in W and B' is a cube in 
the complementary subspace. 

We claim that for sufficiently small k > and all sufficiently large N defined in (|2.7p . 

(2.12) -rL I f 1 (exp(v 1 + Da(z 1 )b)A)f 2 (exp(v 2 + Da(z 2 )b)A)db 

\B\ Jb 

= U hdy\ (J x ^ d f) +0(N^\\h\\ cg \\f 2 \\ c e), 

uniformly over the cubes B and v\,v 2 € C(G), 

Suppose first that (|2,12p holds. Then using uniformity over vi,v 2 , we deduce that 

T77T / fi{exp{Da{z\)u)A) f 2 {exp{Da{z 2 )u)A) du 
>c 



\C\ 

1 



/i(exp(£>a(zi)6' + Da{z 1 )b)A)f 2 {exp{Da{z 2 )b' + Da{z 2 )b)A) dbdb' 



B'\\B\ J B . 

JjldA Uf2dfi) +0(N- K \\f 1 \\ c e\\f 2 \\ c <> 
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Combining this estimate with (|2.10p -( T2.1ip . we obtain 

fo(exp(u)A)fi(exp(Da(zi)u)A)f2(exp(Da(z2)u)A) du 



= I / (exp(«c)A)|C| j (JhdA Uh d » 

+ o((N- K + e e )\\fo\\ce\\fi\\c°\\f2\\c° 
Since / is ^-Holder and (j2.9|) holds, 

^/o(e X p(n c )A)|C|= ]T / /o(exp(u)A) du + O (e*||/« 

CcE CCE C 

/o(exp(n)A) du + O ((e + e e )\\f \\ c o) = J^f du + O (e 9 ||/ ( 



oWc" 



o\\c» 

JE y ' JX K 

Hence, 

(2.13) f f (exp(u)A)f l (exp(Da(z 1 )u)A)f 2 (exp(Da{z 2 )u)A)du 

JE 

Ix f ° dfI ) {Ix fld ^) Ux hdfI ) +0 ( iV ^ ll/o|l ^ ll/l|lce|l/2|lcS )- 
This proves the required estimate when N is sufficiently large, and it is also clear that this 
estimate holds for N in bounded intervals. Hence, Theorem 11.11 follows. Now it remains 
to prove the claim f|2. 12j) . 

To prove (gZSD , we apply Theorem O to the nilmanifold X x X = (G x G)/(A x A) 
with 8 = N~ K . We assume that N is sufficiently large, so that Theorem 12.31 applies. Let 
/ = /l® h- Clearly, 

fd(ii®n)=( I h du\ ( [ fi dfi) and « \\h\\ ce \\f 2 \\ c e. 

XxX \JX J \JX J 



k k \ 

[ + ^2 UDa(zi)wi,v' 2 + ^2 tiDa{z 2 )wi 



We consider the map i : [0,e] fc L{G) defined by 
i{t) = 

with suitably chosen v'i,v 2 € C(G), so that 

/ /i(exp(ui + Da(zi)6)A)/ 2 (exp(w 2 + Da{z 2 )b)A)db = [ f(t(t)A)dt, 

JB J[0,e] k 

It is sufficient to prove that 

e~ k [ f(t(t)A)dt= [ fd{a®^) + 0{5\\f\\ c e). 
J[0,e] k JXxX 

Applying Theorem 12.31 we deduce that either 



(2.14) 



e~ k I f{i(t)A)dt- j /d(/*®/i) 

[0,e] k JXxX 



<8\\f\\ce, 
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or there exists (01,02) E (Z d ) 2 \{(0, 0)} such that 

(2.15) max(||ai||, ||a 2 ||) <C <T Ll = N kL ^ 
and 

(2.16) |(ai, (D^Daiz^Wi) + (a 2 , {D^Da{z 2 ) Wi )\ < S~ L2 /e = N< L * + V 
for all i = 1, . . . , k. 

We shall show that if k > is sufficiently small and N is sufficiently large, then ()2. 15|) — 
(|2,16p fails. Suppose that f)2. 15|) ()2. 16j) holds. Since each of the vectors w x , x G Xq, is a 
linear combination of vectors Wi, we deduce from (|2.16p that 

(2.17) |(ai, (£)7r)£>a(«i)w x ) + (a 2) pvr) J Da(z 2 )w x )| < for a n % G ^ 
Since Da(z)w x = \{ z ) w x an d ^x = Dir(w x ), 17j) becomes 

(2.18) |x(2i)(ai,%)+x(^)(a 2 ,%)| « iV^+i) for all X S -Y . 

We divide the argument into three cases. 

Case 1: suppose that a± = 0. Then a 2 7^ and (a 2 , tD x ) 7^ 0. Moreover, by [U Th. 7.3.2], 

(2.19) I (a 2 ,w x ) I > IKH^ 1 > AT- KLl ( d+1 ). 

By Lemma [2TTI there exists x € A?o such that |x(- 2 2)| > -?V C with fixed c > 0. Hence, it 
follows from OTTBD that 

(2.20) \{a2,w x )\ « iV< i2+1 )- c . 
We assume that the parameter k > satisfies 

(2.21) - KLi(d+ 1) > k(L 2 + 1) -c. 

Comparing (|2.19p and (|2.20p . we get a contradiction if N is sufficiently large. Hence, we 
may assume that a\ 7^ 0. 

Case 2: suppose that a\ 7^ and x( z i) ( a ii^x) + x{ z 2) {0-2, w x ) = for some x € <fo- 
Since the Galois group acts transitively on the set Xq, it follows that this equality holds 
for all x E Xq. By Lemma 12. 11 there exists x € Xq such that |x(z 2 — z\)\ > iV c with fixed 
c > 0. Then 

(2.22) I (a 2 ,w x ) I = |x(zi - 2:2)|| (ai,u) x ) | > iV c | (a u w x ) \. 
Since ai 7^ 0, we have (ai,w x ) 7^ 0, and by [TJ Th. 7.3.2], 

I (ai,w x ) I > ||ai|r d_1 » Ar KLl ( d+1 >. 

On the other hand, 

I («2,^ x ) I <C ||o 2 || <C iV KLl . 

Hence, we deduce that 

We choose the parameter k > so that 

(2.23) -«Li(d+l) + c>KLi. 
Then when iV is sufficiently large, we get a contradiction. 

Case 3: suppose that a\ 7^ and x( z i) ( a i; ^x) + /t( 2; 2) (02, 7^ ^ ^ or au X € <-fo- This 
is the most difficult part of the proof. 
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Since a\ 7^ 0, we have (ai,w x ) 7^ 0, and by [H Th. 7.3.2], 

I (ai,w x ) I » Haxll^- 1 » N - KL ^ d+1 l 

We set u = — |°^ x j ■ By Lemma [2TTT there exists x € Xq such that |x( z i)l 2> A c with 
fixed c > 0. It follows from (|2.18p that for this x-> we have the estimate 

atk(L2+1) 

(2.24) \x(z 2 - zi)u - l| « , -, ,„. a « N <M+Li(d+i))-c_ 

\x(zi)\\ {ai,w x ) I 

Let Ki := k(L 2 + 1 + Li(d + 1)) - c. 

Next, we compare this estimate with the lower estimate provided by Proposition I2.2L 
We note that 



h(u) = n 

max(| {cli,w x ) \ v , I (0*2, w x ) \ v ) 



l/[Q(u):< 



v 



For all non-Archimedian places v, 

I (<k,w x ) \ v < 1, 

and for all Archimedian v, 

I (ai,w x ) \ v < Hoill < A kLi . 

Therefore, 

(2.25) H(u)<A A ' 2 , 

where K2 := kL^ with fixed L' x > 0. We take the parameter k > so that 

(2.26) AT 2 = kL[ < 1. 
Then assuming that iV is sufficiently large, we obtain 

(2.27) log(c 2 H(u)) < \og{c 2 N K2 ) < logiV, 

where c' 2 > 1 depends on the implicit constant in the estimate (|2.25p . We recall that we 
have chosen the indices, so that 

log N = \\z 2 — zi\\. 
Since ()2.27|) holds, Proposition 12.21 applies, and we deduce that 

c 3 \\z 2 - Zl\ 



\x(z2 - zi)u - 1| > exp -ci log(c 2 H(u)) log 



>g(c 2 H(n)), 

Without loss of generality, we may assume that C3 > e. Since the function x i-> x\og(C/x) 
is increasing for x < C/e, we deduce that 

(2.28) \ X (Z2 - Zl )u - 1| > exp (-c x log(^) log ( b gff a) )) 

> exp (- Cl log(c 2 A X2 ) log (cgK,- 1 )) . 
Comparing (12.24j) and (12.28|) . we conclude that 

(2.29) K' 2 log N + M 2 < K x log N + Mi, 
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where K' 2 := — C\K 2 log^K^ 1 ), M 2 : = — c\ log^) logfaK^ 1 ), and M\ is determined 
by the implicit constant in (12. 24ft . We observe that as n — > + , we have K' 2 — > 0~ and 
K\ — > — c < 0. Therefore, taking the parameter k > sufficiently small, we may arrange 
that 

(2.30) K' 2 > Kt. 

Then when N is sufficiently large, (|2.29p fails. This shows that (I2.15p - (|2.16p fails and 
(|2.14p holds when N is sufficiently large. Now we have verified the claim (I2.12P and 
completed the proof of Theorem 11.11 under the irreducibility condition. 

In order to prove Theorem II .11 in general, we observe that using the same argument, one 
can deduce the following more general version of the estimate (j2. 12|) : for all sufficiently 
large N defined in ([27 



(2.31) J fi(hiPi(exp(vi + Da{ Zl )b))k)f 2 {h2^Mv2 + Da{z 2 )b))K) db 

Jjidl^j (J f 2 dA +0(N- K \ 



ll\\c e \\J2\\c e , 



uniformly over the cubes B, hi,h 2 G G, v±,v 2 £ £(G), and automorphisms f3i,(3 2 of G 
which act trivially on G/G'. Indeed, 



/i(Mi(exp(i;i + Da{ Zl )b))A)f 2 (hMeMv2 + Da{z 2 )b))K) db 
frih! exp^i^i) + D^Da(zi)b)A)f 2 (h 2 exp(Df3 2 (v 2 ) + Df3 2 Da(z 2 )b))A) db, 
and to prove (|2.3ip , we can apply Theorem 12.31 to the map 

/ k k \ 

t : t h-> ^1 + 5^ tiDf3\Da{z\)wi, v 2 + ^ tiD(3 2 Da(z 2 )wi 

V i=l i=l / 

As in the above proof, either (|2.31|) holds, or an analogue of (|2.16|) holds, but since 
DirDfii = Dtt, this reduces to the exactly same estimate as (|2.16|) . Therefore, (|2.31l) 
follows. Now we combine (|2.3ip with the argument (|2.8p - ([2.13p to deduce that 



(2.32) / ^ fo(x)fr(h 1 f3 1 (a(z 1 )(x)))f 2 (h 2 l3 2 (a(z 2 )(x))) d^x) 

[ fodi?) U h d p) (J x hdfJ ) +°( N ~ K6 \\fo\\c4h\\c4f2\\ce) 

uniformly over hi , h 2 G G and automorphisms /3i , (3 2 of G that preserve A act trivially on 
G/G'. We will use this estimate to establish Theorem 11.11 in general. 

2.6. 3-mixing in general. Let W be the Lie subalgebra of C(G) introduced in Section 
12.51 By [121 Ch. 5, Sec. 5], there exists a closed connected normal subgroup M of G such 
that Mj (M n A) is compact, and 



exp(W)gA = MgA for almost every g G G. 
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Since we may replace the lattice A by its conjugate, we assume that 



exp(W)A = MA. 

We note that the group M satisfies the following properties: 

(i) M is a(Z')-invariant, 

(ii) Dtt(W) is not contained in a proper rational subspace of £(M/M'), 

(iii) [G, M] C M'. 

Properties (i)~(iii) can be verified exactly as in the proof of [9j Lem. 3.4]. 

We give the proof of Theorem 11.11 using induction on dimension of X. For this, we use 
that X = G/A fibers over Y = G/{MA) with fibers isomorphic to 

R = MA/ A ~ M/(M n A). 

The invariant measure on X can be decomposed as 



fdti= / f(,yr)dfi R (r)dfi Y {y), feC(X), 
x Jy Jr 

where fly and hr are normalised invariant measure on Y and R respectively. Since the 
fibration is a(Z z )-equivariant (by (i)), 



(2.33) / f (x)fi(a(z 1 )x)f 2 (a(z 2 )x)dfi(x) 

fo(yr)fi(a(zi)(y)a(z 1 )(r))f 2 (a(z2){y)a(z2)(r))dfj, R (r) ) d(j, Y (y) 

J 

Mgr)fi(a(zi)(g)a(zi)(r))f 2 (a(z 2 )(g)a(z2)(r)) dfi R (r) ) dm F (g), 



x 



Y \JR 



R 



where F C G is a bounded fundamental domain for G/(MA), and rap is the measure on 
F induced by [iy. We shall show that for defined in (|2.7p and some rj > 0, 

(2.34) f f (gr)f 1 (a(z 1 )(g)a(z 1 )(r))h(a(z 2 )(g)a(z 2 )(r))df, R (r) 



f (gr)dfi R (r) \ [j fi(a(zi)(g)r)dfi R (r) \ [j ^f 2 (a(z 2 )(g)r)dfi R {r] 

+ 0(N^\\f \\ c o\\f l \\ c o\\f 2 \\ c e) 

uniformly over g 6 F. 

Suppose that (|2.34p holds. Then combining (|2.33j) and (|2.34[) . we obtain 



x 



Y 



fo(x)fi(a(z 1 )x)f 2 (a(z 2 )x) dfi(x) 

/o(!/)/i(a(«i)y)/2(a(^2)l/) d/iy (y) + OCJNr^ll/oUc* ll/2|lo»), 

where the functions fi on Y are defined by 

y ^ fi(yr)d/j, R (r). 
JR 
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Since dim(Y) < dim(X), it follows from the inductive assumption that for some 77 > 0, 
fo(y)fi(a(z 1 )y)f 2 (a(z 2 )y) d^ Y (y) 

fodiXY^il fid^il f2d^ Y ) +0(N^\\f \\ c e\\f 1 \\ c o\\f 2 \\ c e) 



Y 

fod^ (J hdA (J f^A + °( N ~ r '\\fo\\c'\\fi\\c<\\f2\\c°), 

and this completes the proof of Theorem II .li Hence, it remains to prove (|2.34|) . 
To prove (|2.34p . we write 

a(zi)(g) = CLirriiXi with a, G F, mi G M, \ G A, i = 1, 2. 

Then 

/ /o(5 f r)/i(a(2 : i)(ff)a(^i)(? , ))/2(a(22)(9)a(^2)(r)) eZ/^r) 
/o(^)/i(aimi/3i(a(zi)(r)))/2(a 2 m2^2(a(22)(r)))d/i jR (r), 

R 

where fa's are the maps induced by the automorphisms m 1— > AjmA" 1 . We observe that 
because of (ii), W C C(M) satisfies the irreducibility assumption of Section [231 and by 
(hi), the automorphisms act trivially on M/M' . Hence, (|2.32[) holds. We apply (|2.32l) 
to the functions on R defined by 



(r) ■= fo(gr) and <^(r) := /i(a;r), i = 1,2. 
This gives 

4>o (r)4>i (mift (a{zi)r))4> 2 (m 2 (3 2 {a{z 2 )r)) dfi R (r) 



R 



>o\\c<>\m\\cf>\m\\c<>, 



fo(gr) d^ R {r) J I J ^h{oi{zi){g)r) dfx R (r) \ \J ^f 2 (a{z 2 )(g)r) d^i R {r) 
+ 0{N^\\h\\ c e\\h\\ c e\\f 2 \\ c e). 



This implies (|2.34p and completes the proof of Theorem II. U 



3. Higher order mixing 

The aim of this section is to prove Theorem 11.21 We shall use notation introduced in 
Section 12.11 In Section 13.11 we prepare Diophantine estimates. Then in Section 13.21 we 
give a proof of Theorem 11.21 under an irreducibility condition, and in Section 13.31 we give 
a proof in general using an inductive argument. 

We note that it is sufficient to prove Theorem ll.2l for a collection of functions fi G L°°(X) 
which is dense in L 1 (X). Hence, we may assume that /o, . . . , f s G C e {X). Furthermore, 
we may assume that zq = 0. 
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3.1. Diophantine estimates. Let K be a number field and S a finite set of places of K 
containing all the archimedean places. We denote by Us the ring of S'-units, namely, the 
groups of elements x in K such that \x\ v = 1 for v £ S. For a vector x, £ K s , we define its 
(relative) height by 

H(x) = Y[ max(l, 



where v runs the set of all places of K, and || 

Proposition 3.1. Let v G S and hi, . . . b s G i^\{0}. Then for every e > ; the inequality 



(3.1) 



J=2 



< H(x)- £ , 



has finitely many solutions x E Us such that no proper subsum ofb\ + X^j=2 ^j x j vanishes. 
We call such solutions of (|3.ip nondegenerate. 

Proof: We prove the proposition by induction on s. Note that when s = 1, the statement 
holds trivially because there are only finitely many solutions of H(x) < c. 

Given a solution x of (|3.ip . we set y = (l,x), and we denote by j v = j v (x) the first 
index j v such that 

ls/j„ k = \\y\\v > i- 

Partitioning the set of solutions according to the index j v , we may assume that this index 
is fixed. 

We introduce a family of linear forms L w j(y), with w G S and j = 1, . . . , s, defined by 

L wj(y) = Vj, (w,j)^{v,j v ), 

s 

L wj(y) = ^2bjyj, (w, j) = (v, j v ). 
Then if y = (1, x) corresponds to a solution of (13. ip . 

s s 

Y\\L wj {y)\ w = Y[\yj\ w , w^v, 

s s 

]J \L V j(y)\v = \L V j v (y)\v n \yj\v < H(y)~ € Yl \Vj\v, 

3=1 jftv 3=1 

and by the product formula, 

s 

(3-2) J];|L wi (y)U <H(y)" e . 

we5 j=l 

By the W. Schmidt subspace theorem [U Cor. 7.2.5], all the solutions of (I3.2p are contained 
in a finite union of proper linear subspaces of K s . Partitioning solutions of (|3.ip according 
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to theses subspaces, we may assume that these solutions additionally satisfy a nontrivial 
linear relation 



(3.3) 



3=2 



with ci, . . . , c s € K. 

Suppose that c\ ^ 0. Given a solution x of (13. 3|) . we pick a minimal J C {2, . 
that 



s} such 



(3.4) 



0. 



Then no proper subsum in (13. 4h vanishes. It follows from the finiteness of the number of 
non-degenerate solutions of unit equations [U Th. 7.4.2] that Xj, j G J, varies over a finite 
set. This shows that for every solution x of (13.3)) there exists jo = 2, . . . , s such that xj 
belongs to a fixed finite set. Hence, in order to prove finiteness of nondegenerate solutions 
(|3.ip . we may assume that xj is fixed. Then (13.ip becomes 



(3.5) 



0>i + b jo x jo ) + b J 

3^30 



< H(£)- 



Since we are assuming that no proper subsum in (13. ip vanishes, b\ + bj Xj ^ and no 
proper subsum in (|3.5h vanishes as well. Let x' = (xj : j ^ jo). Then H(x') < H(x). 
Hence, by the inductive assumption, the number of nondegenerate solutions x' of (|3.5p is 
finite, and this implies the proposition in this case. 

Now suppose that c\ = in (I3.3p . One of C2, . . . , c s is non-zero, and for simplicity, we 
assume that c s ^ 0. Then combining (|3.ip with (|3.3p . we obtain that 



(3.6) 



s-l 



bi + ^(bj - cjb s c s 1 )i 
i=2 



< R(x)' 



Given a solution x of ()3.6|) . we pick a minimal Jc{2 ,s — 1} such that 



(3.7) 



< H(x)" 



and no proper subsum of 6i + ^2j e j{bj — Cjb s c~ 1 )xj vanishes. Let x! = (xj : j G J). Since 
H(x') < H(x), it follows from the inductive hypothesis that x' belongs to a fixed finite 
set. This proves that for every solution x of (|3.ip there exists jo = 2, . . . , s such that xj 
belongs to a fixed finite set. Now we can finish the argument as in the previous paragraph, 
and this completes the proof of the proposition, o 
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3.2. Higher order mixing under irreducibility condition. We define the subspace 
W in C{G), the set of characters Xq and the eigenvectors w x with x £ as in Section 



In this section we assume that Dir(W) is not contained in any proper rational subspace. 
Let {wi, . . . , Wk} be a fixed basis of W. Consider a box map 



where B = [O.Ti] x ■ ■ ■ x [0, T k }. 

Lemma 3.2. Let fi,...,f a G C(X), u\, . . . ,u s G C{G), ■■■ ,/3 s be automorphisms of 
G such that = id of G/G' , z\,...z s G Z , v\, . . . ,v s G £{G), and x\, . . . , x s El. Then 

]W\J B ^ri/i(exp(^)A(a(^)(exp(^ + i(t))))a;i)^ dt = fl( < J x ^ d ^) + o(1) 

as minjUzill, \\zi~ Zj\\ : i ^ j} — > oo. Moreover, the convergence is uniform overui, . . . ,u s , 
Pi,...,j3 s , Vi,...,v s> x 1? ...x s , and functions fi,...,f s with \\fi\\ c e < 1- 

Proof: It is sufficient to prove the claim when fi,-.-,f s belong to a dense family of 
functions in C{X). Hence, without loss of generality, we may assume that the functions 
are Holder with exponent 9. 

To prove the lemma, we apply Theorem 12.31 to the product nilmanifold X s = G s /A s . 
Suppose that the claim of the lemma fails. Then there exist S G (0, <5q) and sequences 



f?>, • • • , f s n> G C e (X), <\ . . . , u s n > G C{G), (3[ n >, PT 1 satisfying ^ = id on G/G', 
z^ , . . . , z a G l) satisfying 



1231 



k 




i=l 




V 



(n) 



vT } G C(G), and x 




G X such that 



l > 



l > ■ ■ ■ j 



5 



1 




We set /< n > = f[ n) <&>••• <g) /i n) : X s -> R, «W = (uj n) , • • • , u£ n) ) G C{G) S , 

: £ -> : 1 1 — y (DP^Daiz^Xv^ + . . . , Z)^^a( Z W)(^ + t(t))) 

and = (xi n) , . . . , xi™^) G X s . Then 



s 



n/ w ii .<nn/* iic. 



i=i 
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and 



1 



/ (n) (exp(uW)exp(^ n )(t))x (n) )dt- / f^d^ 8 

Jx 



»<5||/ (n) ||c*- 



\B\jb 

It follows from Theorem 12.31 that there exists (a^ n \ . . . , ai"^) 6 (Z d ) s \{0} such that 

(3.8) "" (n) ' 
and 



\a\ '||,...,||a<™)|| <<T Ll < 1, 



(3.9) 



^(af ,DnD^ n) Da(z^)( 

3=1 



<^ L2 /Ti<l for all * = !,...,*;. 



Since j3j = id on G/G', we have DirD(3j l> = Dir. We rewrite (I3.9P in terms of vectors 
w x , x € Xq that satisfy Da(z)w x = x{ z ) w x f° r z e ^ ■ Since each w; x can be written as a 
linear combination of tOj's, it follows from (|3.9p that 



,(n) 



(3.10) 



3=1 



< 1 for all x € <*b- 

We observe that because of (|3.8p . the tuple (a^ , . . . , a^) varies over a finite set. Hence, 
passing to subsequence, we may assume that (I3.10P holds for a fixed tuple (ai, . . . ,a s ) E 
(Z rf ) s \{0}. After changing indices, we may assume that a,j ^ for j = 1, . . . s' and <2j = 
for j > s'. We note that this implies that 

bj := (aj, Dtt(w x )} ^ for all j = 1, . . . s' and % S ^o- 

Indeed, if (oj, Dir(w x )) = for some j and x, then taking Galois conjugates we obtain 
that {aj,Dir(w x )) = for all x E <^o- This implies that Dir{W) is contained in a proper 
rational subspace and contradicts the irreducibility assumption. 

We may cancel vanishing subsums from (I3.10p . and passing to a subsequence, we may 
assume that no proper subsum in (I3.10p vanishes. 

Passing to a subsequence and changing indices, we may also assume that 

(3.11) max{||zj n) ||:l<j< s '} = ||4 n) ||. 
By Lemma l2.lt there exists fixed c > such that 

(3.12) max|x(z)| > exp(c||z||), z € Z'. 

Hence, passing to a subsequence, we may assume that 

|Xo04 n) )l > exp(c||^ n) ||) 
holds with a fixed xo G .^o- For this xo> (|3. 10p gives 



(3.13) 



b o + Yl h i x 3 

3=1 



CO 



<C exp(— c\\z\ 
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where Xj := Xo{z^ — z[ ). It is clear that 6j's and Xj are .S-units in a fixed number 
field, and to derive a contradiction, we apply the estimate of Proposition ^. 11 We observe 
that there exists Cy > 1, v 6 S, such that 

\Xo(z)\v < exp(c„||z||), z£Z l . 



Hence, 



and by ([3TTTT) . 



= \xo(zl n) - z[ n) )\ v < - z{ n) \\), 

H($W) = nmax(l,|xi n) |„...,|x^|„) 
V^c„ I max ||z,- r 

/ Kj<s' 3 



< exp 



y (»») » I 



^ ex p ( 2 (l>) ii^ n) n) 



It follows from f)3. 13[) that 
(3.14) 



(») 



6 + 5>, 

i=i 

with fixed e > 0. According to our construction, no proper subsum in (13. 14ft vanishes. 



<H(x (n) )" e . 



Hence, it follows from Proposition 13.11 that = Xo{ z j ~ z i ) runs over a finite set. 





Since all elements in Xq are conjugate under the Galois action, it follows that x( z ^ — z± ) 



with x £ also runs over finite set. In particular 

» » 



On the other hand, by (f3~T2|) . 



max Yu «1. 



i / (n) (n)\\ 

max \y(z- — z\ ) — > oo. 
xe* J 1 



This contradiction proves the lemma, o 



Now we prove Theorem 1 1 . 2 1 under the irreducibility condition. Without loss of generality, 
we may assume that zq = 0. We fix a fundamental domain F C G for G/A and set 
E = exp~ 1 (F). We may arrange that E is bounded and has piecewise smooth boundary. 
Then 



x 



fo(x) ^f[fi{a(zi)(x))j dfi(x) = ^/ (exp(u)A) (j[f t (exp(D 



a(zi)u)A) du, 



where du denotes a suitably normalised Lebesgue measure on C{G). We choose a basis 
of C{G) that contains the fixed basis {wi} of W and tessellate C{G) by cubes C of size e 
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(3.15) 



E-(jC 

CdE 



« e. 



For all u\,U2 6 C, 

(3.16) |/ (exp(m)A) - / (exp(n 2 )A)| « e e . 

Here and later in the argument the implied constants may depend on the Holder norms 
of /o, . . . , f s - For every cube C, we pick a point uc £ C. Then it follows from ()3. 15[) and 
([inSD that 

(3.17) Jj (eM^) (jlfi(eMDa(zi)u)A)j du 
I /o(exp(n)A) (f[ fi(exp(Da{zi)u)A) ) + O(e) 



CC-E ° \i=l 



= £ / (exp(n c )A) / IJJ/iCexp^aCzi^A)) du + 0(e e ). 

C<zE J° \i=l J 

Each cube C in the above sum can be written as C = B' + B where B is a cube in W and 
£?' is a cube in the complementary subspace. 
Let 

N = N(z 1: ...,z s ):= mindl^H, \\z { - zj\\ : i ^ j). 
It follows from Lemma 13.21 that 



(3.18) jijjf ^fiiexpivi + DaiziXb)^ db^f[^f t d^ 

as N — >• oo, uniformly over ui, . . . i> s € £(G) and the cubes i? (note that all the cubes are 
translates of a fixed cube). Hence, it follows that 



l -J c (j[h(eMDa(z l )( 



\C\ 

1 



\B'\\B\ J B 



B 



u))K) du 



\ fi(ex-p(Da(zi)(b') + Da( Zi ){b))A) j dbdb' 



i=i v /J < 
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as N — > oo. Combining this with (|3.17p . we deduce that 

/o(exp(u)A) Y[ fi(exp(Da(zi)u)A) du 
E \i=i J 

= ( £ /o(exp(«cr)A)|C|) f[( f fid?) + ( £ \C\ ) o(l) + 0(e e ), 

\CcE / i=l ^ X ' \Cc£ / 

where clearly ^cc-E 1^1 = ^(-O- Moreover, using (|3.16p and (|3.15p . we deduce that 

/o(exp(«c)A)|C| = E ( / /o(exp(n)A)dn + 0(|C|e e ) 

C<ZE CCE ^ C 

f {exp(u)A)du + O{e 9 ) 



= [ f dfi + O(e e ). 
Jx 

This implies that 

(J\fi(a(zi)(x))j dfi(x) = / (exp(u)A) m ^(exppa^iOA) j 



f[(J x fidp) +o(l) + 0(e 6 



i=0 

as iV — >• oo. This proves Theorem 11.21 under the irreducibility condition. It is clear from 
the proof that convergence is uniform provided that H/ollc 6 ; • • • > ll/sllc 6 ^ 1- 

3.3. Higher order mixing in general. We will apply an inductive argument which uses 
the result of Section [3.21 as a base case. In fact, we note that the argument in Section 13.21 
implies that 

(3.19) f f (x) (f[fi(hiPi(a(zi)(x)))\ dfi(x) =f[( f V) +o(l) 

Jx \ i=1 J i=0 \Jx J 

as N = N(zi, . . . , z s ) — y oo, uniformly over functions fo, ■ ■ ■ , fs with ||/i||ce <C 1, hi, . . . , h s G 
G and automorphisms /3i, . . . ,/3 s of G that preserve A and act trivially on G/G' . Indeed, 
Lemma 13.21 implies that in (|3.18p we more generally have 

JB-\J B ^JiM(exp(vi + Da(zi)(b)))A)j db = f[ (jf f t d^j + o{\) 

as N — > oo, uniformly over fo, ■ ■ ■ , fs with ||/j||c0 <C 1, hi, . . . ,h s G G, automorphisms 
fii, . . . , f3 8 , and vi, . . .v s G C(G). Then the rest of the argument carries over and implies 

Let W be the Lie subalgebra of C(G) introduced in Section [231 By [191 Ch. 5, Sec. 5], 
there exists a closed connected normal subgroup M of G such that M/(MC\A) is compact, 
and 

exp(W)gA = MgA for almost every g G G. 



MIXING PROPERTIES OF COMMUTING NILMANIFOLD AUTOMORPHISMS 23 

Since we may replace the lattice A by its conjugate, we assume that 



exp(W)A = MA. 
We note that the group M satisfies the following properties: 

(i) M is a(Z')-invariant, 

(ii) Dir(W) is not contained in a proper rational subspace of C(M/M'), 

(iii) [G,M] C M' . 

Properties (i)-(iii) can be verified exactly as in the proof of [9l Lem. 3.4]. 

To apply induction, we observe that the nilmanifold X = G/A fibers over the nilmanifold 
of Y = G/(MA) with fibers isomorphic to R = MA/A ~ M/(M n A). The invariant 
measure pnl decomposes as 



[ fdfi = [ [ f(yr)dMr)dtiY(y), feC(X), 

JX JY JR 



where \xy and \x R denote the normalised invariant measures on Y and R respectively. It 
follows from (i) that the fibration X — > Y is a(Z')-equivariant. Hence, we obtain 

(3.20) jj Q {x) m_fMzi)(x)^j dn(x) 

fo(yr) Y[fi{a(zi){y)a{zi){r)) d^ R {r) dfiy(y) 



Y \JR 



\i=l 



fo(9 r ) ( Y\ji{ a { z i){9)a{zi){r)) j dfJ, R (r)j dm F (g), 

where F C G is a bounded fundamental set for G/(MA), and mp is the measure on F 
induced by [iy. We write 

a(zi)(g) = airriiXi with a« G F, nii € M, Aj € A, i = 1, . . . , s. 

Then 



J R fo(gr) ^n/i(a(«i)(5)«(«i)(r))J d m (r) 



R 



fo{gr) ^Y[fi(aimi(3i(a(zi)(r)))j dfi R (r), 

where /3j's are the transformations of S induced by the automorphisms m i— > A^toA^ 1 of 
M. By (ii), the subspace W C C(M) satisfies the irreducibility assumption of Section |3T2"1 
and by (iii), the automorphisms f3i act trivially on M/M'. Let 

4>o{r) ■= fo(gr) and ^(r) := /i(«ir), i = l,...,s. 

Since F C G is bounded, we have H^iH^e <C ||/j||c fl ^ 1- Hence, it follows from (|3.19p 
that 



<Po{r 

R 



) \JlMmPiHzi)(r)))j d t i R (r)=p^y^ i d f x R ^ +o(l) 
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as N — > oo, uniformly over g G F, mi, . . . , m s G M, and the automorphisms /3i,...,/3 s . 
Since OjMA = a(zi)(g)MA, this implies that 



(3.21) 



R 



fo(gr) fa fiWziMa(zi)(r)) \ d m (r) = f[ (^J J^z^r) d m {r^j + o(l) 



as N — > oo, uniformly over g € F. Let be the function on Y defined by 

V >->■ / fi(yr)dfj, R (r). 
JR 

Combining (|3.20p and (|3.2ip . we deduce that 

J x fo(x) (jlfiHziXx))] d^{x)= Jj Q {y) (jljMz^y))] d^ Y {y) + o{\) 
as N — > oo. Finally, it follows by induction by dim(AT) that 

f y My) m /<( a (^)(f ))1 = n (X /< + °w 

The above argument implies uniform convergence provided that ||/o||c e > • • • > ll/sllc 9 ^ 1- 
This completes the proof of Theorem 11.21 



4. Exponential mixing of shapes 

While we have proved exponential 2-mixing and 3-mixing for Z'-actions by automor- 
phisms on nilmanifolds, we do not know if exponential mixing of higher orders holds for 
them in general. This would require to establish a quantitative version of Proposition ^. II 
which seems to be out of reach of available number-theoretic methods. Nonetheless, we 
prove a weak form of exponential mixing where the error term is controlled by 

N*(zq, . . . ,z 8 ) := min min {\x{*i ~ z j)\ ■ * + j} 
xeAf(a) \x(Zi-Zj)\>l 

with notation as in Section |2. 11 This, in particular, implies exponential mixing for Anosov 
shapes — Theorem 11.31 

Theorem 4.1. Let a : 7) -> Aut(X) be an action on a compact nilmanifold X such 
that every a(z), z ^ 0, is ergodic. Then there exists n = r](0) > such that for every 
/o, •..,/« € C e (X) and z , . . . , z s e I 1 , 



(4 
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Proof: We adapt the method of the proof of [HI Th. 1.2] from our previous paper. Since 
the proof is quite similar to the argument in this paper, we will only give an outline. 

We take a character x € X{a) and the corresponding eigenvector w £ C(G). If x 
is real, we denote by W the corresponding one-dimensional eigenspace. Otherwise, we 
denote by W the two-dimensional subspace (w,w) fl C(G). Then Da(z)\w = r(z) • oj(z) 
where r(z) = \x( z )\ an d w(z) is a rotation. We assume in addition that x 2 ^ X{a). Then 
W is closed under the Lie bracket. 

We first treat the irreducible case: namely, when Dir(W) is not contained in a proper 
rational subspace. Without loss of generality, N* > 1. Then for all i ^ j, \x( z i — Zj)\ 7^ 1 
and after changing indexes, |x( z o)| < |x(^i) | < ... < |x(z s )|. We may also assume that 
zq = 0. We fix a basis of C(G) which contains a basis of W and tessellate C(G) by cubes 
of size e with respect to this basis. Then the integral 



can be approximated by a sum of the integrals 

/ ^n/i(exp(I?a(zi)w) A )^ du 

with the error of size 0(e e Y\i=i ll/llc 9 )- Since C = B' + B where B is a cube in W and 
B' is a cube in the complementary subspace, the above integral can be written as 




~[fi(exp(Da(zi)b' + Da( Zi )b)A) j dbdb'. 



_ e -dim(W) 



For every cube B, we take a box map lb '■ [0, e] dim (W) — ^ B that parametrises B. Then by 
Prop. 4.2], there exists k > such that 

J/i(exp(uj + r(zi)u(zi)i B {t))A) dt 
i=i / 

=n(X /4 ^) +o ( cj ~ K n ||/||ce ) 

uniformly over € where <7 = min{er(zi), r(z s )/r(z s _i), . . . , r(z2)/r(zi)}. 

We note that the Diophantine condition for the box map t i-» co(zi)LB(t), which is required 
in [9j Prop. 4.2], is satisfied because is spanned by vectors with algebraic coordinates 
and [TJ Th. 7.3.2] applies. Since this estimate is uniform over Uj's, it follows that 



[0 i£ ]dim(WO 



]4 / (n/i(exp(Da(^)«)A)) du = f[(f hd^) + O (a' K f[\\f\\ 

' ' u/c \i=l / i=l ^ x ' V i=l 



c > 
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and we deduce that 

f x /o(z) (fl fiWzi)*)] dfi(x) = II (/^ fi ^) + O ({e e + a~ K ) f[ \\f\\ c ej . 

We refer to the proof of [9l Th. 1.2] for details. Choosing e = r(z±)~2 implies the claim of 
the theorem in the irreducible case. 

To give a proof in general, we use induction on dim(X). This argument is very similar 
to Section 12.61 If exp(W)A ^ X, we consider the a-equivariant fibration X — > Y defined 
by the closure. The above argument implies that 

f x Hx) ^n/ifaA(a(*)*))j d M*o =n +o y N * v p.y\\cej 

uniformly over hi, . . . ,h s € G and automorphisms /3±, . . . , /3 S G Aut(X) that act trivially 
on G/G' . (In fact, this uniformity was part of [91 Prop. 4.2].) Then as in Section [2.61 

f x Hx) ^J[fMzi)x)^ dn(x)= J^foix) ^flfMzi)x)j d» Y (x) + 0\N-«n\\f\\ cS j 
where /o, . . . , f s € C e (Y). Now the claim follows by induction on dimension, o 

Remark 4.2. In the irreducible case of the above proof, we can replace N*(zo, . . . ,z s ) 

Ni(zo, . . . ,z 8 ) := max min {\x(zi - Zj)\ : i ^ j}, 
xeA "(a) \x(Zi-Zj)\>l 

which provides a better estimate. 

Proof of Theorem \1.3\ : We note that if in Theorem 14.11 we assume that a(zi — Zj) are 
Anosov for all i ^ j, then N*(zq, ■ ■ ■ , z s ) > 1 and N*(nzo, . . . , nz s ) = N^(zq, . . . , z s ) n . 
Hence, Theorem 11.31 follows directly from Theorem 14.11 

5. COCYCLE RIGIDITY 

We now apply exponential 2-mixing to prove smooth cocycle rigidity for genuinely 
higher-rank abelian actions by automorphisms of nilmanifolds — Theorem 1 1.51 The proof 
is based on the "higher-rank trick" from |10j . 

Let ll -> Aut(X) be an action on a compact nilmanifold X and c : l) x X — > R a 
cocycle. Assume that there is a rank-two subgroup (a, b) of ll such that its every nonzero 
element acts ergodically on X. 

First, we note that the map cq : ll \— > R defined by z h > J x c(z, x) d[i{x) is a homomor- 
phism by the cocycle property. Then c — Co is also a cocycle, and it will suffice to prove 
the theorem for c — cq. Thus, we will assume that all functions c(z, •) for zeZ 1 satisfy 
f x c(z, x) dfi(x) = 0. 

Let f(x) = c(a,x). We shall show that there exists cf> G L 2 (X) such that 



(5.1) 



f = <j>oa-</>. 
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We will apply our previous results (9j Sec. 6]. By [9l Theorem 6.1], it suffices to show that 

oo „ oo 

a 2 := / 2 *|2V (foa i )fd P L= V (foa\f) = 0. 

Jx i=l Jx l = ~OC 

We note that the assumption of this theorem is verified in (9J Sec. 6] using exponential 
mixing of a. Let h{x) = c(b> ,x). It follows from the cocycle property that 

fob 3 — f = h o a — h. 

Hence, 

n n 

(/ ° -foa i )= J^{ho a i+1 -hoa i ) = ho a n+l - h o aT", 

i=—n i=—n 

and it follows from exponential mixing that 

a 2 = (f o- 1 ^ , f) f° r every j G Z. 

l=— oo 

On the other hand, by the exponential mixing for the group (a, b) established in Theorem 

m 

]T (foa i V,f)<oc. 
i,jez 

This implies that a 2 = and proves (|5.ip . 

Now using the cocycle regularity result [21 Theorem 7.1] established in our previous 
paper, we deduce that (|5.ip also has a C°° solution. Finally, since a acts ergodically, it 
follows from |10|. Lem. 4.1] that 

c(z, •) = (j) o z — (j) for all z G Z'. 

This completes the proof of Theorem 11.51 
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